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THIS DIGEST WAS CREATED BY ERIC, THE EDUCATIONAL RESOURCES 
INFORMATION CENTER. FOR MORE INFORMATION ABOUT ERIC, CONTACT 
ACCESS ERIC 1 -800-LET-ERIC 

This digest is based on time proven guidelines developed for use in training workshops 
for state and local educators to outline the processes by which performance 
assessments could be created, validated, and used in large-scale assessment. The 
term performance assessment as used in this digest is reserved primarily for those 
assessments that go beyond paper-and-pencil, group-administered assessments. This 
type of assessment is an important and unique tool available for measuring student 
performance at the state or local level. 

These guidelines are provided to offer guidance to district and state policy makers and 
assessment directors concerning some of the issues of managing the development, 
administration, and use of performance assessments in large-scale assessment 
programs. The following sections present information about how performance 
assessments can be developed, administered, scored, and reported. The paper 
suggests informal, less costly means to develop and use these types of measures. 
However, some of the uses outlined above might require the use of external contractors 
or technical advisors. 

PRE-ASSESSMENT DEVELOPMENT ACTIVITIES 



Before assessment development can or should occur, several important planning 
activities must set the stage for assessment. These steps take place at the outset to 
insure that the assessment is developed in a manner that fits the content area to be 
assessed and is within the resources available. These steps include the development of 
the assessment framework, creation of the assessment plan, determination of 
assessment resources, and production of the assessment blueprint. 

The framework for the assessment serves as the guide to the entire assessment. This 
consists of the assessment objectives. It is from the framework that the assessment is 
developed. There are many ways to determine these objectives. The traditional manner 
in which to do this is to gather a group of content area experts and classroom teachers 
and ask them to indicate what students should know and be able to do by the end of 
certain points of instruction. However, an alternate procedure is to allow the framework 
of expectations to evolve out of research and practical experience of educators on what 
outcomes students are capable of at particular times in their school career. 

The assessment plan provides an overview and description of the types of assessment 
to be developed and used, as well as the manner in which assessments will be 
implemented. The plan serves the useful purpose of describing the types of 
assessments that are envisioned and how assessments will be administered, scored, 
and reported. The assessment plan should force the agency sponsoring the 
assessment development to carefully consider the resources needed and available in 
order to make decisions about the assessment at the outset. Once the assessment plan 
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has been written and the assessment resources have been determined, it is possible to 
make any adjustments needed in the assessment plan. 

The blueprint will describe the characteristics of an adequate assessment for each 
content area of the assessment framework as well as the characteristics of an adequate 
assessment for each student outcome. Once completed, this should guide the 
development of the assessments that are needed given the resources available. The 
final step in the pre-assessment development activities is for the assessment plan and 
the assessment blueprint to be approved by the sponsoring agency and any additional 
advisory groups or individuals. 

ASSESSMENT DEVELOPMENT STEPS 



After the blueprint has been created, it is time to formulate the assessment prompts. 
Throughout this process the developer must consider several things. These include: 1) 
in what manner should the assessment questions or instructions be presented to the 
student, 2) what additional stimulus materials will be needed, 3) how the students will 
respond and how such responses will be recorded and scored, 4) what criteria will be 
used to judge student responses, 5) the number of scale points for scoring student 
responses, and 6) samples of each level of response. 

Once the exercises have been written, the next step is to edit them. This is an important 
step in assuring that the exercise and assessment administration processes are 
understandable, that the exercise warrants the special time and attention which 
performance measures require, and that there is consistency among the performance 
exercises. The editor also can check to ensure that the stimulus materials fit with the 
exercises. 

One of the more difficult aspects of performance exercises is writing the assessment 
administration directions. Many performance exercises are administered to students 
individually or in small groups. These exercises require the assessment administrator to 
set up a standard situation to which each student can respond, as well as, to read a 
standard set of directions to each student. Developing standard assessment 
administration directions that are complete and accurate is usually the result of trying 
the exercise out one or more times and noting areas of students confusion, responses 
that students provided which are vague or incomplete, and ways in which some or all of 
students responded that were not anticipated. If a majority of student responses indicate 
that the assessment instrument is ineffective, considerable restructuring of the testing 
tool may be in order before any further testing of the instrument is completed. 

Once the sample student responses are gathered, someone needs to review the 
responses and attempt to score them according to the criteria and preliminary scoring 
guide developed by the exercise writer. After the initial work on the scoring guide is 
complete, an expert panel of judges should be convened. This panel should be asked to 
review each exercise, confirm the preliminary judgments for each score scale point, and 
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discuss those student responses that did not appear to be scorable according the 
preliminary scoring guide. This panel may note changes that needed to be made in the 
assessment administration process. Following the institution of these changes, the new 
test should be retested and reviewed by the panel for final approval. 

PREPARATION FOR ASSESSMENT 
ADMINISTRATION 



The next important set of activities is getting ready for the actual assessment. This 
involves selecting the schools that will participate, preparing the schools for participation 
in the assessment, and training the individuals who will gather the data from students. 
Since the assessment materials which have been developed and refined are intended 
for large-scale assessment use, it is presumed that the exercises will be given to some 
or all of the students at one or more grade or age level. The original assessment plan 
will help to determine how and to whom the exercises will be administered. This 
information will aid in the decision of which schools are selected to participate in the 
standardized performance-based assessment. Letters of notification should be sent to 
the school coordinator and to the district coordinator if applicable. These letters should 
include such details as when the assessment will occur, what will be involved in the 
assessment, what the schools responsibilities will be, who will administer the test, and 
so forth. 

A major key to the success of the entire performance assessment project is the quality 
of the individuals who are chosen to conduct the assessment. After the selection 
process for new assessment administrators has taken place, training should occur. 
When the assessment administrators are comfortable with the administration of the 
tests and the manipulatives and equipment that are used in them, the focus of the 
training is to shift to recording and scoring of responses. Trainees with acceptable 
scoring prowess will be certified as assessment administrators; others must be either 
cycled back through more training or dismissed from the assessment project. 

ASSESSMENT ADMINISTRATION 



Once the assessment administrators are trained, they also should contact the 
appropriate assessment coordinators. The purpose of this contact is to schedule the 
date(s) and times for the assessment administration to occur, to remind the school 
coordinator to have a complete listing of students at the appropriate grade level(s) 
available on assessment day, and specify what facilities and equipment will be needed. 
After all contacts have been made, the field assessment administrators can begin the 
process of assessment administration. This will start with the drawing of the sample of 
students to be assessed and a list of alternates. It may be helpful to have the school 
assign an aide or a student who can work with the assessment administrator to locate 
the students when needed for the assessment and bring them to the assessment 
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administration site. 

It will also be most helpful if the designated contact person and others associated with 
the project but not involved directly in the assessment administration would select some 
schools in which to observe one or more students taking part in the performance 
assessment. In addition to observing the assessment administration, these people can 
also discuss the assessment with some of the students following the assessment. This 
can provide valuable insight on why and how students responded and their motivation 
and interest in the assessment. 

POST- ASSESSMENT ADMINISTRATION 
ACTIVITIES 



Various student responses are selected by an expert in the area to represent the 
different types of responses that students may have given. Such expert judgments are 
next confirmed by an expert panel of judges; some of the sample will appear in the 
scoring guide prescored and will be used to train the scorers. The others will be used to 
judge the accuracy of the scorers following the initial training. 

The scoring process requires a number of things to be prearranged for the scoring to 
flow smoothly. First, a determination would have to be made about whether there will be 
one or two scorers for each response. Second, arrangements will need to be made to 
distribute booklets and other materials to be scored, plus rating sheets, to each of the 
scorers. Third, routine reliability checks should be built into the scoring process. 

The various scores need to be summarized and prepared for reporting. The summaries 
will be most efficient if the individual(s) who will be doing the reporting and those people 
directing the scoring discuss the needed and desired data summarization process 
before the scoring is conducted. One thing to keep in mind when reporting the data is 
that typical audiences are most interested in knowing how well students performed, why 
students performed as they did, if the experts were surprised in any way by the level or 
types of student performances, and what the experts believe needs to be done to help 
students improve. 

CONCLUSION 



While performance assessment may be new to some people, it is not new nor is it 
untried. National and statewide performance assessments were successfully conducted 
in a reliable and cost-efficient manner decades ago. As this guide has illustrated, 
performance assessment is feasible and manageable. Such assessments are vitally 
needed in the assessment landscape so that those interested in assessing what 
students are capable of doing have access to more complete information on student 
performance. Although the steps are more complex and more involved, such 
assessments are important in the determination of what skills our students need to have 
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and whether or not they do in fact have them. Performance assessment is an important 
adjunct to overall large-scale assessment strategies. 
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